Pre- and post-shutter signal image capture and sort for digital camera

ABSTRACT

A camera system includes an image sensor and a processing apparatus receiving image frames from the sensor before and after receipt of a user picture signal. The processing apparatus discards frames that do not meet a quality criterion such as under-exposed frames. Also, the processing apparatus compresses only a subset of remaining frames, specifically, those that meet a compression amount threshold. The remaining frames are presented to a user, who can select a representative frame as the “picture” that was taken.

This application claims priority from U.S. provisional patentapplication Ser. No. 61/190,991, filed Sep. 3, 2008.

I. FIELD OF THE INVENTION

The present invention relates generally to capturing images in a digitalcamera both prior to and after the user generates a shutter open signal,and then sorting the images for the user to select one or more images.

II. BACKGROUND OF THE INVENTION

Digital camera users often miss the opportunity to capture an image atthe right time. This is sometimes due to the slow camera response ofpoint-and-shoot digital cameras. Another potential reason is that thesubject has moved, wrongly posed, framing is not correct or lighting haschanged, resulting in a not correctly exposed image. As a consequence,users are often disappointed to realize that the end result is not thedesired one when they review their captured images either on the deviceor on their personal computer.

SUMMARY OF THE INVENTION

A method includes, prior to receiving a take picture signal from acamera user input element, capturing plural digital image frames. Themethod also includes, after receiving the take picture signal, capturingplural image frames. The captured frames can be considered asestablishing an initial set of frames. Frames not satisfying a qualitycriterion are discarded from the initial set of frames to render afiltered image set. The filtered image set is presented to a user forselection of one or more images therefrom, and responsive to a userframe selection, one or more frames from the filtered image set arestored.

In some implementations the method may include compressing only framesyielding a minimum compressed size to establish a compressed set offrames. Only frames in the filtered image set may be compressed. Framesin the filtered image set may not be compressed if the frame results ina compressed file larger than a threshold. Only frames in the compressedset of frames may be presented to the user.

The method may be executed in a digital camera device and/or in acomputer receiving frame information from a digital camera device.

In another aspect, a camera system has an image sensor and processingapparatus that receives image frames from the sensor before and afterreceipt of a user picture signal. The processing apparatus discardsframes that do not meet a quality criterion and compresses only a subsetof remaining frames for presentation thereof to a user.

In another aspect, a system includes means for producing image framesprior to and after generation of a picture signal to render an initialset of frames. Means are provided for filtering frames from the initialset of frames based on at least one quality metric to render a filteredset of frames. Also, means are provided for presenting frames from thefiltered set of frames to a user for selection of at least one frame.

The details of the present invention, both as to its structure andoperation, can best be understood in reference to the accompanyingdrawings, in which like reference numerals refer to like parts, and inwhich:

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic block diagram of an example camera; and

FIG. 2 is a flow chart of example logic in accordance with presentprinciples.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring initially to FIG. 1, a digital camera 10 includes a portablelightweight hand-held housing 12 holding a camera sensor 14 such as butnot limited to a charge-coupled device (CCD). The sensor 14 producesimages sensed through a lens 16, and in the example non-limitingembodiment shown the images are sent to a processing circuit 18 whichexecutes compression/frame selection logic in accordance with disclosurebelow, it being understood that the logic of FIG. 2 alternatively may beentirely embodied by software on, e.g., the below-described PC orstorage medium.

In turn, the processing circuit 18 may communicate with a cameraprocessor 20 in the housing 12, which can access and store data on acomputer-readable medium 22. The medium 22 may be, without limitation,disk-based storage and/or solid state storage and in one implementationis implemented by random access memory (RAM).

To activate the camera 10, a power button 24 may be provided on thehousing 12. A user can manipulate the power button 24 to cause one ormore camera batteries 26 to energize the components of the camera 10,including the processor 20 and a visual display 28 such as but notlimited to a liquid crystal display (LCD). Also, a picture button 30typically is provided on the housing 12 that can be manipulated by auser to signal the user's desire to capture a frame as a picture. Ifdesired, a communications interface 32 such as but not limited to auniversal serial bus (USB) interface may be provided to enable thecamera processor 20 to communicate with, e.g., a user's personalcomputer 34.

It is to be understood that the camera 10 may be implemented as anelectronic device with an imaging sensor and storage such as digitalcameras per se, camera-equipped mobile phones, personal digitalassistants (PDAs), and notebook computers with built in cameras.

Now referring to FIG. 2, at block 35 the camera 10 is activated by,e.g., the user manipulating the power button 24. In the case of a mobilecomputing device equipped with a camera block 35 may be implemented bythe launch of a camera application.

At block 36 images are captured and filtered-compressed as set forthfurther below, and at block 38 image capture continues after receipt ofa user-generated “take picture” signal as might be generated by the usermanipulating the picture button 26. In one embodiment, “N” frames arecaptured after receipt of the “take picture” signal, wherein “N” is aninteger; also, if desired only “M” frames (wherein “M” is an integer)prior to the “take picture” signal may be retained.

In any case, proceeding to block 40 the frames that preceded the takepicture signal and that were retained in, e.g., a temporary location inRAM along with the frame taken in response to the take picture signalplus the post-signal frames that were retained are analyzed for quality,with frames not satisfying one or more quality metrics being discarded.For example, if a frame is found to be under-exposed, i.e., exposedbelow a threshold, such a frame may be discarded at block 40, thussaving storage space and bandwidth.

As understood herein, the image sequence captured as described above canbe compressed at block 42 using preferably lossless video codingtechnology to minimize storage requirements, and additionally onlycertain frames may undergo compression. Specifically, at block 42 onlyframes that result in a minimal compressed file may be compressed, i.e.,only frames that satisfy a compression amount threshold of, e.g., atleast 20% compressed are compressed. For example, a frame that mighthave passed the exposure test at block 40 but nonetheless due toexposure anomalies would result in a significant increase in the size ofthe compressed file because motion compensation techniques that are usedin lossless video coding might fail to find a frame similar to the frameunder test and, thus, result in a relatively large file even aftercompression, would not be compressed at block 42. Thus, only a subset offrames surviving the screening test of block 40 might be compressed atblock 42.

The final compressed set of frames from block 42 is stored at block 44and presented (after decompression) to the user at block 46 on, e.g.,the display 28. The frames may be presented automatically or in responseto user signals, which may allow the user to scroll through the framesand then select one or more, with non-selected frames being discardedfrom memory if desired.

In alternate embodiments instead of executing the above-describedprocessing entirely on the camera 10, frames may be sent to a PC orserver through the communication interface 32 for processing asdescribed. Also, users not only have the opportunity to analyze capturedimage sequence on the camera 10 itself, but alternatively may downloadthe frames output by block 42 to the PC 34 and use a softwareapplication that allows them to browse the image sequence and re-capturethe desired frame.

In the example non-limiting implementation shown, the lossless videocoding algorithm is implemented in a hardware processing circuit 18 thatdirectly captures images from the sensor 14, with the camera 10processing and storing the compressed image sequence to the storagemedium 22 using direct memory access (DMA) to minimize processingoverhead. However, present principles can be implemented in software ineither the camera 10 or the personal computer 34.

If desired, image enhancements in accordance with image enhancingprinciples known in the art can be applied to the image sequence beforebeing presented to the user. Such enhancement can include but are notlimited to applying super-resolution algorithms to the captured frames.

It may now be appreciated that using present principles, users ofdigital cameras or camera equipped mobile devices have the opportunityto correct problems in captured images by being given the chance toreview not only their captured image but also the previous and followingframes too. Users can select the frame of their choice either on theirdevice or on their personal computer and finally save it to their album.They also have the opportunity to create mosaics or enhance the qualityof the captured image by using super-resolution imaging on the capturedimage sequence.

In addition to the above, various sensors may be used determine when tostart automatic frame capturing. Usage of various sensors such asorientation sensors, heat sensors, camera CCD/CMOS images, proximitysensors, or accelerometers can be used to determine when to capture apicture. For example, the above-described automatic capturing can beginas soon as a proximity sensor detects that the user's face is near theviewfinder eyepiece.

Additionally, metrics such as image contrast or proper exposure may beused to determine if a frame that has been captured automatically isworth keeping or not. Other metrics that may be used are determinationsof whether points of interest are located/distributed properly, whetherthe color/light is properly balanced, etc.

Furthermore two or more frames may be combined using, e.g.,super-resolution techniques to create a new picture that is acombination of a few sub-optimal frames and that is much better than anyof the original frames, in effect a frame that the user might never havebeen able to capture otherwise. Super-resolution imaging may thus beused to improve dynamic range and also eliminate camera sensor noiseduring shooting a night scene.

While the particular PRE- AND POST-SHUTTER SIGNAL IMAGE CAPTURE AND SORTFOR DIGITAL CAMERA is herein shown and described in detail, it is to beunderstood that the subject matter which is encompassed by the presentinvention is limited only by the claims.

1-7. (canceled)
 8. Camera system comprising: image sensor; andprocessing apparatus receiving image frames from the sensor before andafter receipt of a user picture signal, the processing apparatusdiscarding first frames not meeting at least one quality criterion basedon a determination that the first frames do not meet the qualitycriterion, the processing apparatus further deleting second frames thatdo not satisfy a minimum compression threshold based on a determinationthat the second frames do not compress to a size small enough to satisfythe compression threshold, the processing apparatus compressing only atleast some frames other than the first and second frames forpresentation thereof to a user.
 9. Camera system of claim 8, wherein theprocessing apparatus includes processing circuitry within a camerahousing.
 10. Camera system of claim 8, wherein only a maximum of “N”frames imaged after the user picture signal and a maximum of “M” framesimaged prior to the user image signal are considered for compression.11. Camera system of claim 8, wherein the quality criterion issufficient exposure.
 12. (canceled)
 13. Camera system of claim 8,wherein the processing apparatus is external to a camera housing holdingthe image sensor.
 14. System comprising: means for producing imageframes prior to and after generation of a picture signal to render aninitial set of frames; means for filtering first frames from the initialset of frames based on a determination that the first frames fail tosatisfy at least one quality metric; means for deleting second framesfrom the initial set of frames based on a determination that the secondframes, if compressed, would be larger than a compression threshold; andmeans for presenting frames not filtered by the means for filtering andnot deleted by the means for deleting to a user for selection of atleast one frame.
 15. System of claim 14, further comprising: means forcompressing frames not filtered by the means for filtering and notdeleted by the means for deleting.
 16. System of claim 15, wherein thecompressing means and filtering means are embodied in a camera housingholding the means for producing.
 17. System of claim 15, wherein thecompressing means and filtering means are embodied in a computerseparate from the camera housing.
 18. System of claim 14, wherein thequality metric includes exposure.
 19. System of claim 14, wherein themeans for filtering is embodied in hardware.
 20. System of claim 15,wherein the means for filtering and means for compressing are embodiedin software.